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Analysis  of  Reservation-Based  Dual-Link  Networks 
for  Real-Time  Applications 


Abstract:  Next-generation  networks  are  expected  to  support  a  wide  variety  of  services.  Some 
services  such  as  video,  voice,  and  plant  control  traffic  have  explicit  timing  requirements  on 
a  per-message  basis  rather  than  on  the  average.  In  this  paper  we  develop  a  general  model 
of  reservation-based  dual-link  networks  to  support  real-time  communication.  We  examine  the 
desirable  properties  of  this  network  and  the  difficulties  in  achieving  these  properties.  We  then 
introduce  the  concept  of  coherence  and  develop  a  theory  of  coherent  dual-link  networks.  We 
show  that  a  coherent  dual-link  network  can  be  analyzed  as  though  it  is  a  centralized  system.  We 
then  discuss  practical  considerations  in  implementing  a  dual-link  network,  and  implications  of 
this  work  to  address  problems  observed  in  the  IEEE  802.6  metropolitan  area  network  standard. 


1  Introduction 


Real-time  communication,  defined  as  communication  with  explicit  timing  requirements,  is  im¬ 
portant  to  future  networks  which  will  concurrently  support  a  wide  variety  of  sen/ices.  Examples 
include  multimedia  traffic,  such  as  digital  audio  and  digital  video;  and  real-time  computing  traf¬ 
fic,  such  as  plant  process  control  and  air-traffic  control  systems.  In  traditional  applications  of 
packet-switched  networks,  performance  is  measured  by  average  throughput  and  average  re¬ 
sponse  time.  However,  guaranteed  timing  performance  is  needed  for  real-time  communication. 
The  desirable  properties  of  a  network  that  supports  real-time  communication  include: 


Predictable  Operation:  By  predictable  we  mean  that,  given  an  arbitrary  set  of  network  con¬ 
nections,  we  can  predict  if  timing  constraints  of  all  the  connections  can  be  met. 

High  Degree  of  Schedulability:  Schedul ability  is  the  degree  of  network  utilization  at  or  below 
which  individual  message  timing  requirements  can  be  insured.  It  can  also  be  thought  of 
as  a  measure  of  the  capability  of  supporting  timely  connections. 

Position-Independent  Bandwidth  Allocation:  The  amount  of  bandwidth  allocated  to  a  sta¬ 
tion  must  be  position-independent  and  under  protocol  control. 

Stability  Under  Transient  Overload:  When  the  network  is  overloaded  and  it  is  not  possible 
to  meet  each  connection’s  timing  requirements,  more  critical  connections  must  meet  their 
timing  requirements  at  the  expense  of  less  critical  connections. 


It  may  not  be  easy  to  achieve  the  above  properties,  as  evidenced  by  the  problems  of  IEEE  802.6 
metropolitan  area  network  standard,  as  discussed  by  several  researchers  [vAWZ90,  CGL91 , 
SS90].  Scheduling  in  a  network  is  different  from  scheduling  in  a  centralized  environment.  In  a 
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centralized  system,  all  resource  requests  are  immediately  known  to  the  centralized  scheduler. 
In  a  network,  distributed  scheduling  decisions  must  be  made  with  incomplete  information.  From 
the  perspective  of  any  particular  station,  some  requests  could  be  delayed  and  some  may  never 
be  seen,  depending  on  the  relative  position  of  the  station  in  the  network.  The  challenge  is  to 
achieve  predictability  under  these  circumstances. 

In  this  paper,  we  develop  an  analytical  model  of  reservation-based  dual-link  networks  and  use 
it  to  reason  about  the  relationship  between  bandwidth  requests  on  one  link,  and  the  patterns 
of  slot  usage  by  stations  on  the  other  link.  The  resulting  model  of  slot  usage  serves  as  a 
foundation  for  studying  the  behavior  of  dual-link  networks.  We  shall  use  this  model  to  analyze 
the  schedulability  of  periodic  traffic  and  propose  possible  solutions  to  problems  observed  in 
IEEE  802.6. 

The  remainder  of  this  paper  is  organized  as  follows:  In  Section  2  we  describe  the  architecture 
and  operation  of  dual-link  networks.  We  discuss  the  difficulties  in  scheduling  traffic  in  dual-link 
networks  and  introduce  the  concept  of  system  coherence.  Section  3  discusses  a  proposed 
protocol  for  media  access  control  that  results  in  both  coherent  operation  using  a  coherent  reser¬ 
vation  protocol  (CRP)  and  regulated  access  of  the  media  through  the  flow  control  protocol.  In 
Section  4  we  analyze  the  behavior  of  coherent  systems  and  develop  results  about  the  relation 
between  slot  reservation  patterns  in  coherent  systems.  Section  5  discusses  the  scheduling  of 
periodic  traffic  in  a  dual-link  network  that  follows  CRP  and  flow  control.  We  introduce  the  notion 
of  transmission  schedulability  for  a  dual-link  networks  and  show  that  connections  in  a  coherent 
dual-link  network  are  transmission  schedulable  if  they  are  schedulable  in  a  centralized  sys¬ 
tem.  Section  6  discusses  practical  considerations  in  implementing  the  conceptual  model  of  a 
dual-link  network,  and  the  implications  of  this  work  to  the  addressing  unpredictability  observed 
in  the  IEEE  802.6  DQDB  (distributed  queue  dual  bus)  protocol.  Section  7  makes  concluding 
remarks  and  discusses  future  research  directions. 


2  Conceptual  Framework 


In  this  section,  we  first  review  the  basic  architecture  of  a  reservation-based  dual-link  network 
as  discussed  in  the  IEEE  802.6  standard  [Sta90].  However,  we  develop  the  bandwidth  reser¬ 
vation  abstraction  using  transmission  queues  in  stations,  instead  of  counters,  since  counters 
are  simply  an  efficient  implementation  of  queues.  We  then  introduce  the  concept  of  system 
coherence  as  a  basis  for  predictability  of  dual-link  networks. 


2.1  Architecture  of  Dual-Link  Networks 

A  dual-link  network  consists  of  two  slotted  unidirectional  links,  say  Forward  Link  (Flink)  and 
Reverse  Link  (Rlink),  as  shown  in  Figure  1 .  Fixed-length  slots  are  generated  by  slot  generators 
of  the  corresponding  links.  Although  the  figure  shows  slot  generators  as  separate  functional 
units,  the  slot  generation  function  can  be  embedded  in  stations  at  the  end  of  the  links.  Each 
station  is  able  to  transmit  and  receive  messages  on  both  links.  We  assume  that  each  message 
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slot 


Flink 


Figure  1 :  Dual-Link  Network  Design 

is  partitioned  into  one  or  more  packets,  and  exactly  one  packet  can  be  transmitted  in  a  slot. 
We  assume  that  a  station  wants  to  send  a  number  of  messages  to  another  and  call  this  a 
connection  between  the  stations.  In  a  high-speed  metropolitan  area  network,  the  slot  delay 
is  small  compared  to  the  network  delay.  To  simplify  discussion,  we  will  use  the  slot  delay  as 
the  unit  of  measurement.  We  therefore  assume  that  each  slot  is  transmitted  in  unit  time  and 
stations  are  separated  by  an  integral  number  of  slot  times.  Stations  reserve  slots  on  Flink  by 
making  requests  on  Rlink.  Since  the  delay  for  transmitting  a  single  slot  is  small  compared  to 
network  propagation  delay,  in  this  paper  we  ignore  the  slot  delay  by  assuming  that  stations 
wish  to  make  requests  synchronously  with  the  arrival  of  Rlink  slots.  In  the  following  discussion, 
we  will  only  discuss  message  transmissions  on  Flink  and  reservation  on  Rlink,  because  of 
symmetry  in  the  network.  Referring  to  Figure  t,  stations  on  the  right-hand  side  are  called 
downstream.  Stations  on  the  left-hand  side  are  called  upstream. 

Each  Flink  slot  contains  a  BUSY  bit  to  indicate  whether  or  not  the  slot  is  used.  BUSY=0  indi¬ 
cates  an  empty  slot.  A  node  may  transmit  in  an  empty  slot  by  setting  BUSY=1  and  copying  its 
packet  into  the  slot.  However,  with  BUSY  bits  alone,  stations  closer  to  the  Flink  slot  generator 
can  monopolize  the  link.  To  minimize  this  positional  priority,  stations  use  a  REQ  bit  in  Rlink 
slots  to  reserve  Flink  slots. 

Before  discussing  the  reservation  mechanism,  we  describe  Rlink  slots  and  introduce  an  ab¬ 
straction  that  will  allow  us  to  analyze  if  each  station  gets  the  Flink  slots  it  requested.  Since 
we  discuss  only  Flink  transmissions  in  this  model,  slots  on  the  Rlink  are  used  only  to  reserve 
Flink  Bandwidth.  Therefore  Rlink  slots  carry  request  information  consisting  of  the  presence 
of  a  request  and  its  priority.  In  our  abstraction  we  represent  an  Rlink  slot  to  contain  an  REQ 
bit  and  a  priority  field.1  In  addition,  we  imagine  that  the  Rlink  slot  contains  a  field  to  hold  the 

’There  may  be  alternate  ways  to  communicate  a  request’s  priority  level.  For  example,  in  IEEE  802.6  a  separate 
REQ  bit  is  used  for  each  priority  level,  as  discussed  in  Section  6. 
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Figure  2:  Slots  on  Rlink  with  REQ  Bit,  RSID,  and  Priority  Fields 


requesting  station’s  address  (RSID),  as  shown  in  Figure  2.  The  RSID  field  is  not  part  of  the 
implementation  and  is  used  only  to  facilitate  analysis.  If  the  REQ  bit  is  set,  the  associated 
combination  of  the  RSID  and  priority  fields  is  defined  as  a  request. 

Flink  slots  can  be  considered  to  contain  only  a  BUSY  bit  and  data  in  this  model.  Furthermore, 
we  introduce  the  concept  of  assignment  of  Flink  slots  to  a  station.  That  is,  when  an  Rlink 
slot  arrives  at  the  head  station,  the  next  Flink  slot  is  said  to  be  assigned  to  the  station  that 
made  the  request.  However  the  head  station  continues  to  release  slots  even  if  there  are  no 
Rlink  requests.  These  slots  are  called  unassigned  slots.  Assigned  and  unassigned  slots  are 
abstractions  that  we  will  use  in  the  analysis. 

A  model  of  each  station  in  the  network  is  shown  in  Figure  3.  Each  station  contains  two  sets 
of  queues.  For  requests,  a  station  contains  a  prioritized  outgoing  request  queue  that  is  used 
for  holding  pending  requests  in  priority  order.  A  station  which  wants  to  make  a  high-priority 
request  can  preempt  a  lower-priority  request  on  the  Rlink  and  replace  it  with  its  high-priority 
request.  The  preempted  request  is  inserted  in  the  station’s  outgoing  request  queue  in  priority 
order. 

For  transmission  on  the  Flink,  each  station  contains  a  prioritized  transmission  queue.  When¬ 
ever  requests  pass  the  station  on  the  Rlink,  they  are  inserted  into  the  transmission  queue  in 
priority  order.  For  each  unoccupied  slot  on  the  Flink,  the  station  dequeues  one  request  from 
the  top  of  the  transmission  queue.  In  addition,  there  are  additional  buffers  that  are  used  for 
flow  control  purposes,  which  will  be  discussed  in  Section  3. 

The  dual-link  architecture  abstraction  provides  us  with  a  convenient  vehicle  to  reason  about 
properties  of  a  dual-link  network.  Finally,  it  is  important  to  note  that  distributed  scheduling 
decisions  with  incomplete  information  are  unavoidable  in  a  dual-link  network.  Some  requests 
made  by  stations  are  seen  by  other  stations  after  a  propagation  delay,  while  some  requests 
may  never  be  seen.  For  example  in  Figure  1,  station  S,+1  does  not  see  requests  from  station 
Su  and  Si  sees  requests  from  stations  5l+fc  after  some  delay.  The  challenge  is  to  achieve 
predictability  under  these  circumstances. 
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Figure  3:  Station  Model  Using  Priority  Queue 

2.2  The  Concept  of  System  Coherence 

To  address  the  distributed  scheduling  problem,  we  describe  the  fundamental  concept  of  system 
coherence.  Intuitively,  coherence  is  a  logical  and  orderly  relationship  between  elements  of  a 
system.  In  the  context  of  dual-link  networks,  the  relationships  that  make  a  system  coherent 
are:  losslessness,  consistency,  and  bounded  priority  inversion.  In  the  following  discussion,  we 
define  each  of  these  concepts. 

Definition  1  Lossless  System: 

A  prioritized  reservation  system  is  said  to  be  lossless  if  and  only  if  each  request  from 
downstream  stations  is  registered  correctly.  That  is,  a  station  copies  each  passing  re¬ 
quest  from  Rlink  without  error  or  loss. 


Definition  2  Consistent  System: 

A  prioritized  reservation  system  is  said  to  be  consistent  if  and  only  if  the  queues  of  re¬ 
quests  in  different  station  queues  are  consistent  with  each  other.  That  is,  if  request 
and  request  R2  both  exist  in  queue  Qa  and  queue  Qb,  and  if  R\  is  ahead  of  R2  in  Qa,  then 
R i  must  also  be  ahead  of  R2  in  Qb. 

Note  that  in  Figure  1 ,  if  requests  from  station  S2  are  not  correctly  registered  by  Si ,  station 
Si  may  not  let  unoccupied  slots  go  by,  and  S2  may  be  unable  to  transmit  and  meet  its  timing 
requirements.  Therefore  it  is  easy  to  see  that  lossless  queues  are  necessary  for  predictable 
operation.  Some  additional  concepts  that  bind  slot  types  to  requesting  stations  are  necessary 
to  illustrate  the  importance  of  consistent  queues.  We  therefore  defer  the  discussion  to  Section 
4.2,  where  we  give  an  example  to  show  that  inconsistent  queues  lead  to  unpredictability. 
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In  an  idealised  priority  scheduling  system,  a  high-priority  request  will  never  need  to  wait  for 
lower-priority  traffic.  In  a  real  system,  a  higher-priority  request  may  have  to  wait  for  lower- 
priority  messages,  The  duration  of  such  waiting  is  known  as  priority  inversion  [SRL90].  Priority 
inversion  interferes  with  the  operation  of  priority-based  scheduling  [SRL90].  For  a  system  to 
be  predictable,  the  worst-case  priority  inversion  must  be  bounded  by  some  function  so  that  its 
impact  can  be  taken  into  account  in  the  analysis. 

Definition  3  Bounded  Priority  Inversion: 

A  prioritized  system  is  said  to  suffer  from  priority  inversion  if  higher-priority  activity  can 
be  delayed  by  lower-priority  activity  [SRL90].  The  duration  of  priority  inversion  is  said 
to  be  bounded  with  respect  to  the  network  size  if  the  delay  is  not  larger  than  2k D,  where 
k  is  an  arbitrary  weight  and  D  is  the  end-to-end  network  propagation  delay. 

An  example  of  unbounded  priority  inversion  is  given  in  Section  3.1.  We  will  show  that  in  a 
coherent  dual-link  network  Jfe=1  and  priority  inversion  is  bounded  by  2D. 

In  summary,  we  define  a  coherent  system  as  follows: 

Definition  4  System  Coherence: 

A  system  is  said  to  be  coherent  if  it  has  the  following  properties: 

•  It  is  a  lossless  system. 

•  It  is  a  consistent  system. 

•  Priority  inversion  is  bounded. 

In  the  next  section,  we  discuss  the  conditions  that  a  system  must  satisfy  to  achieve  the  above 
properties. 


3  Media  Access  Control 

In  this  section,  we  will  consider  two  protocols  to  control  access  to  the  dual-link  network.  First 
we  describe  a  protocol  for  making  reservations  for  slots  on  the  Flink.  Then  we  describe  a  flow 
control  protocol  that  regulates  the  use  of  Flink  slots. 


3.1  Coherent  Reservation  Protocol 

As  discussed  in  Section  2.2  a  coherent  system  must  be  lossless,  consistent,  and  must  have 
bounded  priority  inversion.  In  this  section,  we  discuss  the  conditions  to  be  satisfied  or  rules 
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to  be  followed  to  achieve  these  properties.  The  system  will  be  lossless  if  the  queues  are 
lossless.  The  system  will  be  consistent  if  the  self-entry  rule,  and  the  tie-breaking  rule  (described 
below)  are  followed.  The  system  will  have  bounded  priority  inversion  if  the  station  has  priority 
queues,  requests  on  the  Rlink  can  be  made  autonomously,  and  the  lower-priority  requests  can 
be  preempted. 

Before  considering  the  above  rules  and  conditions,  we  introduce  the  following  notation:  a  re¬ 
quest  by  station  Si  at  priority  p  is  denoted  as  RiP.  When  discussing  requests  of  equal  priority, 
the  second  subscript  is  dropped.  R,p  -<  R:q  denotes  that  Rij  is  "ahead”  of  Rjq. 

A  condition  for  a  lossless  system  is  that  all  requests  on  Rlink  must  be  entered  into  station 
queues  without  loss. 


Condition  1:  Lossless  queues 

The  station  must  be  fast  enough  to  copy  every  request  on  the  Rlink  in  the  observed 
order  without  loss  or  error. 


We  now  consider  the  rules  for  system  consistency.  The  self-entry  rule  defines  the  relative 
ordering  in  which  a  station  must  make  a  self-entry  in  its  transmission  queue  and  a  request  on 
the  Rlink. 

Condition  2:  Self-entry  rule 

A  station  that  wishes  to  transmit  must  make  a  request  on  the  Rlink  before  making 
a  self-entry  into  its  transmission  queue. 

The  following  example  illustrates  the  importance  of  this  rule: 

Example  1  Consider  three  stations  C  and  B  and  A  which  are  at  the  same  priority  and  A  is 
downstream  with  respect  to  B,  which  is  itself  downstream  with  respect  to  C.  Suppose  B  makes 
a  self-entry  ft*,  in  its  transmission  queue  and  then  attempts  to  make  a  request  on  the  Rlink.  Let 
B  be  prevented  by  making  a  request  on  Rlink  by  higher-priority  requests  until  request  Ra  by 
station  A  passes  by.  On  the  request  stream  Ra  ■<  R*  while  in  B's  transmission  queue  it*  -<  Ra. 
After  the  requests  are  registered  in  station  C,  the  transmission  queue  of  C  will  have  Ra  <  R* 
which  is  inconsistent  with  the  queue  of  station  B  as  shown  in  Figure  7. 

Another  rule  for  system  consistency  is  the  tie-breaking  rule,  which  is  designed  to  preserve 
the  ordering  of  equal-priority  requests  on  the  Rlink.  When  a  station  preempts  a  request  at  a 
certain  priority  and  inserts  it  into  its  outgoing  request  queue,  it  must  give  the  preempted  request 
higher  priority  than  other  equal-priority  requests  it  observes  on  the  Rlink.  An  efficient  method 
of  accomplishing  this  is  to  favor  local  requests  over  equal-priority  requests  on  the  Rlink. 
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Condition  3:  Tie-breaking  rule 

•  Preempted  requests  with  equal  priority  are  stored  in  FIFO  order. 

•  When  a  request  local  to  a  station  and  Rlink  requests  have  same  priority,  the 
local  request  replaces  the  Rlink  request  and  the  Rlink  request  is  inserted  in 
the  station’s  outgoing  request  queue. 

The  importance  of  this  condition  is  illustrated  in  the  following  example: 


Example  2  Consider  two  requests  of  equal  priority  A,  and  A,  on  the  Rlink,  such  that  initially 
Ri  -<  Rj.  Let  a  station  S*  preempt  A,  and  replace  it  with  a  high-priority  request  Rb •  Now  5* 
wants  to  make  request  A^.  Let  it  observe  request  Rj.  Since  Ri  and  Rj  are  at  the  same  priority,  it 
cannot  preempt  Rj ,  (if  local  requests  are  not  given  higher  priority  than  equal-priority  requests 
on  the  Rlink),  and  has  to  let  Rj  pass.  Eventually  station  5*  successfully  makes  request  A,. 
Note  that  now  Rj  <  Ri  on  Rlink,  reversing  the  initial  order.  This  reversing  of  the  initial  order 
makes  station  queues  inconsistent. 


We  now  consider  three  conditions  to  achieve  bounded  priority  inversion.  It  is  self-evident  that 
priority  queues  minimize  priority  inversion.  The  other  two  conditions  are  autonomous  request 
traffic  and  request  preemption  property.  We  discuss  each  of  these  conditions  as  follows: 

Condition  4:  Priority-ordered  queues 

All  the  requests  in  each  station’s  transmission  queue  and  outgoing  request  queue 
must  be  in  priority  order.  Equal-priority  requests  are  in  FIFO  order. 


Ability  to  make  autonomous  requests  is  important;  if  stations  are  prevented  from  making  re¬ 
quests  on  the  Rlink  by  traffic  on  the  Flink,  priority  inversions  may  occur.  Suppose  a  station 
cannot  make  a  new  request  if  any  of  its  previous  requests  are  outstanding.  This  results  in  a 
lack  of  autonomy  between  making  requests  on  Rlink  and  the  presence  of  occupied  slots  on 
Flink.  That  is,  requests  from  a  station  are  ’’throttled"  by  traffic  on  the  Flink. 


Condition  5:  Autonomous  requests 

The  request  generation  on  Rlink  is  said  to  be  autonomous  if  each  station  can  make 
its  request  at  the  Rlink  independent  of  the  traffic  at  the  Flink. 


The  dependence  of  the  request  rate  on  Flink  traffic  may  cause  unbounded  priority  inversion, 
as  shown  in  the  following  example: 


Example  3  Consider  a  network  with  two  stations  S^  and  S2,  which  are  d12  slot  times  apart,  as 
shown  in  Figure  4.  Let  station  S,  have  n  slots  to  transmit  every  period  of  100n,  where  n  is 
large  compared  with  d12.  Let  station  S2  generate  real-time  traffic  that  must  be  transmitted  in  1 
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Rlink 


Figure  4:  Effect  of  Non-Autonomous  Requests 

slot  out  of  every  10  slots.  Let  S2  be  assigned  a  higher  priority  than  Si .  Let  the  protocol  require 
that  a  station  cannot  make  a  new  request  if  it  has  an  outstanding  request. 

Let  Si  start  transmitting  first.  Since  it  is  the  only  active  station  on  the  network,  it  transmits  in 
the  first  n  slots  on  the  Flink.  When  S2  desires  to  transmit,  it  will  be  able  to  make  one  request 
on  the  Rlink  and  must  wait  until  its  request  is  satisfied  before  it  can  make  another  request.  The 
request  from  S2  will  reach  Si  after  d12  slot  times.  Then  Si  will  let  an  unoccupied  slot  go  by 
on  Flink  that  will  be  used  by  S2  after  an  additional  delay  of  rf12  slot  times.  Therefore  S2  will 
be  able  to  transmit  once  every  2 d12  slot  times.  However  the  station  wishes  to  transmit  once 
every  10  slots.  Therefore  for  <f12  >  5,  station  S2  will  miss  deadlines  even  though  it  has  higher 
priority  than  Si.  This  occurs  because  S2  is  prevented  from  making  requests  at  a  high  priority 
by  occupied  Flink  slots  even  though  they  are  at  a  lower  priority.  Note  that  the  priority  inversion 
lasts  as  long  as  Si  wishes  to  transmit.  Therefore,  since  transmission  time  of  Si  may  be  longer 
than  2kD  for  any  chosen  k  and  D,  the  inversion  is  unbounded.  Note  that  this  priority  inversion 
occurs  even  if  the  network  utilization  is  as  small  as  11%. 

This  "throttling"  effect  is  implemented  in  IEEE  802.6,  and  behavior  similar  to  this  example  has 
been  observed  [vAWZ90].  Another  effect  of  the  "throttling"  property  is  that  it  can  also  cause 
priority  inversion  among  sources  within  a  station.  Consider  a  station  with  two  sources  at  dif¬ 
ferent  priorities.  If  the  station  has  an  outstanding  request,  it  is  prevented  from  making  another 
request  at  any  priority.  Therefore  a  high-priority  request  may  be  blocked  by  an  outstanding 
lower-priority  request. 

Now  consider  the  example  of  Figure  4  with  the  "throttling"  restriction  relaxed.  Let  Si  start  first  in 
overload  condition  as  before  and  transmit  in  all  slots  on  the  Flink.  When  S2  starts,  it  will  make 
one  request  every  10  slots  irrespective  of  Flink  traffic.  After  an  initial  delay  of  d12  slots,  station 
Si  will  not  use  one  slot  every  10  slots.  The  first  unoccupied  slot  will  reach  S2  after  an  additional 
di2  slot  times.  Therefore  S2  will  be  prevented  from  transmitting  for  an  initial  2<f12  slot  times 
but  thereafter  will  be  able  to  transmit  once  in  every  10  slots  and  meet  its  timing  requirements. 
Priority  inversion  between  sources  within  a  station  will  also  be  avoided.  Since  the  station  can 
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REQ=1  and  q  >  j 


Figure  5:  Station  Operation  Under  CRP 

make  a  new  request  even  when  a  previous  request  is  outstanding,  a  high-priority  source  in  a 
station  is  not  prevented  from  making  a  request  when  a  low-priority  request  is  outstanding. 

Another  condition  for  minimizing  priority  inversion  is  the  request  preemption  rule. 

Condition  6:  Request  preemption  rule 

A  station  which  wants  to  make  a  high-priority  request  can  preempt  a  lower-priority 
request  on  the  Rlink  and  replace  it  with  its  high-priority  request.  The  preempted 
request  is  inserted  in  the  station’s  outgoing  request  queue  in  priority  order. 


The  following  example  illustrates  that  a  lack  of  request  preemption  can  result  in  unbounded 
priority  inversion. 

Example  4  Consider  a  station  5  with  a  high-priority  connection,  and  assume  that  S  wants  to 
make  a  request.  Let  all  downstream  stations  have  lower-priority  connections.  Without  request 
preemption,  the  downstream  stations  can  make  requests  in  all  Rlink  slots  and  thus  indefinitely 
prevent  S  from  making  requests.  This  results  in  unbounded  priority  inversion. 

We  now  propose  a  coherent  reservation  protocol  (CRP)  that  implements  the  conditions  and 
rules  described  in  this  section.  Consider  the  state  diagram  in  Figure  5. 
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Definition  5  Coherent  Reservation  Protocol 

In  the  MONITOR  state,  the  station  copies  each  request  it  sees  on  the  Rlink  into  the  ap¬ 
propriate  position  in  the  transmission  queue.  The  position  depends  on  the  value  of  the 
P  field  of  the  Rslot  that  contains  the  request  When  an  unoccupied  slot  passes  on  the 
Flink,  the  station  dequeues  the  entry  at  the  top  of  its  transmission  queue.  If  the  de¬ 
queued  entry  is  not  a  self-entry,  the  station  lets  the  slot  go  by.  If  the  dequeued  entry 
is  a  self-entry,  the  station  will  also  set  BUSY=1  and  copy  its  packet  into  the  slot.  The 
request  is  then  said  to  be  satisfied. 

If  a  station  intends  to  transmit  at  priority  j  (Ij-\ ),  it  goes  into  the  READY  state  whenever 
it  observes  an  Rlink  slot.  In  this  state  there  are  three  possibilities  to  be  considered, 
depending  on  the  contents  of  the  REQ  and  P  fields  of  the  observed  slot. 

•  lfREQ=  1  and  P  =  q,  whereq  >  j,  then  the  station  goes  back  to  the  MONITOR  state. 

•  If  REQ  =  1  and  P  =  q,  where  q  <  j,  the  station  goes  to  the  PREEMPTREQ  state.  In 
this  state  the  station  replaces  q  in  the  P  field  with  j,  clears  lj=0.  It  also  replaces  the 
contents  of  the  RSID  field  with  its  own  address.  The  preempted  request  is  held  in 
the  outgoing  request  queue  in  priority  order.  The  station  then  makes  a  self-entry 
into  its  transmission  queue,  and  goes  back  to  the  MONITOR  state. 

•  If  REQ  =  0,  the  station  goes  to  the  SETREQ  state.  It  sets  REQ  =  1,  P  =  j,  writes  its 
address  into  the  RSID  field,  clears  lj=0,  makes  a  self-entry  into  its  transmission 
queue,  and  goes  back  to  the  MONITOR  state. 


3.2  Flow  Control  Protocol 

In  this  section  we  discuss  a  flow  control  mechanism.  We  give  an  example  to  show  that  flow 
control  can  be  used  to  bound  the  delay  of  lower-priority  connections  without  affecting  higher- 
priority  connections.  Next  we  describe  the  flow  control  protocol. 

The  delay  between  a  connection’s  request  and  the  arrival  of  its  assigned  slot  is  equal  to  2d,, 
where  di  is  the  distance  between  the  source  station  and  the  Flink  slot  generator.  Since  can 
equal  multiple  periods,  the  connection’s  requests  from  multiple  periods  can  be  accumulated  in 
the  transmission  queue  of  the  source  station  and  all  upstream  stations.  This  creates  the  need 
for  flow  control,  as  illustrated  in  Example  5. 


Example  5  Consider  a  network  with  two  stations,  as  shown  in  Figure  6.  Let  connection  r0  at 
station  S\  be  transmitting  a  message  in  n  slots  every  100n  slots,  where  n  is  large  compared 
with  propagation  delay  (in  slots)  between  Sy  and  S2.  Let  this  message  have  the  lowest  priority. 
Let  station  S2  have  a  connection  r,  to  some  unspecified  downstream  station.  Let  ry  want  to 
transmit  two  packet  in  every  8  slots.  Connection  r,  makes  2  requests  in  every  8  slots  and 
receives  two  assigned  slots  that  it  uses.  Let  a  new  connection  t2  be  established  at  t0  which 
wishes  to  transmit  1  slot  in  every  4  slots.  The  condition  at  time  t0  is  shown  in  Figure  6. 
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Figure  6:  Flow  Control  Example 


Connection 

Num.  Pkts 

Period 

Priority 

TO 

n 

100n 

Lowest  (B) 

Tl 

2 

8 

Medium  (M) 

T2 

1 

4 

High  (H) 

Table  1 :  Connections  on  Network 
The  connections  and  their  relative  priorities  are  shown  in  Table  1 . 

Note  that  after  time  to,  connection  r2  transmits  in  slots  Ml,  which  have  been  assigned  to  r,. 
Therefore  r,  cannot  meet  its  timing  requirements.  Furthermore,  r2  cannot  transmit  in  1  slot  out 
of  every  4.  Therefore  neither  connection  r,  nor  t2  can  meet  its  timing  requirements.  Note  that 
if  connection  r,  had  its  source  at  S,  instead  of  52,  the  new  connection  r2  would  not  cause  it  to 
miss  its  deadlines. 


As  shown  in  the  above  example,  the  establishment  of  a  new  connection  may  prevent  previously 
established  connections  from  meeting  their  timing  requirements,  depending  on  their  positions 
relative  to  the  new  connection.  Note  that  the  total  load  is  only  51  %.  Depending  on  the  param¬ 
eters  of  the  connection  and  the  network,  connections  can  miss  deadlines  even  though  work 
loads  are  arbitrarily  small. 

Suppose  that  a  connection’s  periodic  source  makes  Ci  requests  every  2*  slot  times.  The  objec¬ 
tive  of  flow  control  is  to  ensure  that  no  more  than  Ci  requests  from  the  connection  are  satisfied 
every  T*.  Together  with  system  coherence,  flow  control  provides  a  foundation  for  analyzing  the 
schedulability  of  coherent  dual-link  networks. 

Section  2  discussed  the  station  model  for  dual-link  networks  and  briefly  mentioned  the  need 
for  flow  control  buffers.  We  now  discuss  the  flow  control  architecture  in  detail.  In  the  station, 
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requests  at  each  priority  will  be  inserted  into  a  FIFO  buffer  associated  with  that  priority  level. 
In  addition,  each  buffer  is  associated  with  a  set  of  packet-counters  (PCs),  a  set  of  propaga¬ 
tion  delay  timers,  and  a  set  of  period-timers  (PTs).  One  counter-timer  triple  is  assigned  to  a 
connection  at  that  priority  level. 


Definition  6  Flow  Control  Protocol 

1.  Initialization:  When  the  upper  network  layer  protocol  initiates  a  connection  r  at  sta¬ 
tion  Si  with  periodT  and  number  of  requests  per  period  C,  the  period-timer  is  preset 
to  T,  the  packet-counter  is  preset  to  C,  and  the  propagation  delay  timer  is  preset 
to2di. 

2.  Operation: 

(a)  When  the  first  request  of  r  arrives  at  station  Sit  the  propagation  delay  timer 
starts  counting  down.  The  timer  expires  after  a  time  equal  to  twice  the  propa¬ 
gation  delay  between  the  station  and  the  Flink  slot  generator.  When  the  timer 
expires,  the  request  is  inserted  into  the  transmission  queue.  The  period-timer 
starts  counting  down  and  the  packet-counter  is  decremented. 

(b)  Before  the  period-timer  expires,  whenever  a  request  for  the  connection  ar¬ 
rives  (or  is  present  in  the  flow  control  buffer),  it  is  inserted  into  the  transmis¬ 
sion  queue,  and  the  packet-counter  is  decremented.  If  the  counter  reaches 
zero  before  the  timer  expires,  no  additional  requests  can  be  inserted  into  the 
transmission  queue  until  the  timer  is  reset.  This  ensures  that  no  more  than  C 
requests  are  inserted  into  the  transmission  queue  per  period  ofT. 

(c)  When  the  timer  expires,  both  the  packet-counter  and  the  timer  are  reset  and 
the  process  continues. 

3.  When  the  upper  network  layer  protocol  disconnects  the  connection,  the  above  pro¬ 
cess  continues  when  the  FIFO  buffers  are  emptied.2  The  timer-counter  pair  is  then 
reset  and  made  available  for  new  connections. 


The  above  protocol  can  be  optimized  to  reduce  the  total  number  of  timers  and  counters.  How¬ 
ever  the  optimized  implementation  is  outside  the  scope  of  this  paper. 


4  Analysis  of  Coherent  Reservation  Protocol 


We  now  show  that  the  conditions  described  above  result  in  coherent  systems.  We  assume 
that  a  network  follows  CRP,  and  prove  that  station  queues  are  consistent,  priority  inversion  is 
bounded,  and  the  system  is  coherent. 

2ln  this  paper  we  do  not  consider  abrupt  connection  termination. 
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4.1  System  Consistency 

We  first  show  that  station  queues  in  a  dual-link  network  that  follows  CRP  are  consistent  with 
each  other.  Lemma  1  shows  that  the  order  of  equal-priority  requests  on  Rlink  is  maintained 
Lemma  2  shows  that  the  order  of  equal-priority  requests  in  station  transmission  queues  is  the 
same  as  their  order  on  the  Rlink.  Lemma  3  combines  the  previous  lemmas  to  show  that  equal- 
priority  requests  in  station  queues  are  consistent  with  each  other.  Lemma  4  observes  that  since 
station  queues  are  in  priority  order,  different  priority  requests  in  station  queues  are  consistent. 
Then  by  Lemmas  3  and  4,  Theorem  5  shows  that  the  system  is  consistent. 

Lemma  1  In  a  multi-priority  dual-link  network  that  follows  CRP,  each  station  preserves 
the  order  of  equal-priority  requests  on  the  Rlink. 

Proof: 

Consider  two  equal-priority  requests,  R,  and  Rj,  that  pass  station  S.  Without  loss 
of  generality,  let  Ri  -<  Rj  on  the  Rlink.  We  must  show  that  preemption  does  not 
reverse  the  order  between  Ri  and  Rj  if  the  tie-breaking  rule  is  used.  There  are  only 
the  following  four  cases  to  be  considered: 

Case  1:  Neither  Ri  nor  Rj  are  preempted  by  S. 

In  this  case  the  lemma  is  true  since  no  preemption  occurs  and  Ri  -<  Rj  by  assump¬ 
tion. 

Case  2:  Only  R,  is  preempted  by  5. 

The  order  between  Ri  and  Rj  can  be  reversed  only  if  Rj  passes  S  before  the  station 
can  make  the  preempted  request  Ri  on  the  Rlink.  But  by  the  tie-breaking  rule,  the 
station  favors  preempted  request  Ri  over  Rj ;  therefore,  Rj  cannot  pass  station  S 
if  the  station  is  waiting  to  make  preempted  request  Ri,  and  so  order  reversal  is  not 
possible. 

Case  3:  Only  Rj  is  preempted  by  5. 

In  this  case  the  lemma  is  true  since  Ri  is  not  preempted  and  remains  ahead  of  Rj. 

Case  4:  Both  Ri  and  Rj  are  preempted  by  S. 

In  this  case  both  Ri  and  Rj  will  exist  in  the  outgoing  request  queue  of  Station  S. 

Then  by  the  request  preemption  rule,  the  preempted  requests  are  held  in  the  out¬ 
going  request  queue  in  FIFO  order.  Hence  Ri  <  Rj  in  the  outgoing  request  queue. 
Therefore  by  operation  of  the  CRP  protocol,  R,  and  Rj  will  reappear  on  Rlink  in  the 
order  that  Ri  <  Rj.  □ 
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Lemma  2  In  a  dual-link  network  that  follows  CRP,  for  any  pair  of  equal-priority  requests 
Ri  and  Rj,  IfRi  ■<  Rj  on  the  Rlink,  then  whenever  both  Ri  and  Rj  exist  in  the  same  queue, 
Ri<Rj  in  each  station’s  transmission  queue  and  outgoing  request  queue. 

Proof: 

Due  to  CRP,  station  queues  are  lossless;  that  is,  Ri  and  Rj  will  be  copied  in  the 
transmission  queue  without  loss.  Condition  4  ensures  FIFO  order.  Hence  Ri  -<  Rj 
in  the  transmission  queue. 

Similarly  if  a  station  preempts  both  Ri  and  Rj,  then  R,  <  Rj  in  the  outgoing  request 
queue.  Therefore  the  Lemma  follows.  □ 

Lemma  3  In  a  multi-priority  dual-link  network  that  follows  CRP,  equal-priority  requests 
in  station  queues  are  consistent.  That  is,  for  two  equal-priority  requests  Ri  and  Rj,  if 
Ri  -<  Rj  in  any  station  queue,  then  Ri  <  Rj  in  every  station  queue  where  both  R,  and  Rj 
exist. 

Proof: 

From  Lemma  2,  since  equal-priority  requests  in  all  station  queues  are  consistent 
with  the  order  of  requests  on  the  Rlink,  and  since  by  Lemma  1 ,  the  order  of  requests 
of  equal-priority  requests  is  maintained,  it  must  be  the  case  that  if  Ri  -<  Rj  in  any 
station  queue,  then  Ri  -<  Rj  in  every  station  queue  where  both  Ri  and  it,  exist.  □ 

Lemma  4  In  a  multi-priority  dual-link  network  that  follows  CRP,  different  priority  requests 
in  station  queues  are  consistent.  That  is,  for  two  different  priority  requests  RiP  and  Rjq, 
if  RiV  <  Rjq  in  any  station  queue,  then  RiP  -<  it,,  in  every  station  queue  where  both  Rip 
and  Rjq  exist. 

Proof: 

Let  RiP  -<  Rjq  in  station  5*.  Since  the  dual-link  network  follows  CRP,  station  queues 
are  in  local  priority  order.  Therefore  RiP  is  at  higher  priority  than  Riq.  Since  every 
station  queue  is  in  priority  order,  R,p  x  Riq  in  every  queue  where  both  exist.  □ 

Theorem  5  In  a  dual-link  network  that  follows  CRP,  station  queues  are  consistent  with 
each  other. 

Proof: 

By  Lemma  3,  equal-priority  requests  are  consistent  in  station  queues.  By  Lemma  4, 
different-priority  requests  are  consistent  in  station  queues.  Therefore  the  theorem 
follows.  □ 
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4.2  Bounded  Priority  Inversion  and  System  Coherence 


We  have  shown  that  a  dual-link  network  using  CRP  has  consistent  queues.  To  show  that  the 
system  is  coherent,  we  need  to  demonstrate  that  priority  inversion  is  bounded.  We  begin  by 
establishing  a  relationship  between  requests  on  the  Rlink  and  the  pattern  of  Flink  slot  usage. 
Then  in  Theorem  7  we  show  that  a  request  cannot  be  satisfied  by  a  slot  assigned  to  a  higher- 
priority  request.  In  Theorem  8  we  show  that  priority  inversion  is  bounded  by  the  round  trip 
network  delay.  Finally,  the  combination  of  consistent  queues  and  bounded  priority  inversion 
results  in  system  coherence. 

We  introduced  the  concept  of  Flink  slots  assigned  to  a  station  in  Section  2.  When  a  Rlink  slot 
arrives  at  the  head  station,  the  next  Flink  slot  is  said  to  be  assigned  to  the  station  that  made 
the  request.  However  the  head  station  continues  to  release  slots  even  if  there  are  no  Rlink  re¬ 
quests.  These  slots  are  called  unassigned  slots.  The  importance  of  this  assignment  abstraction 
is  that  if  each  station  were  to  use  only  its  assigned  slot,  it  would  be  possible  to  determine  the 
worst-case  slot  usage  patterns  by  stations.  We  show  later  that  coherent  systems  do  exhibit  the 
above  behavior.  First,  we  will  show  that  an  incoherent  system  exhibits  unpredictable  behavior 
depending  on  the  location  of  unassigned  slots. 

Example  6  Consider  the  network  in  Figure  7,  with  three  stations,  say  A,  B  and  C,  that  wish  to 
transmit  at  the  same  priority.  The  slot  generator  has  assigned  slots  to  station  requests  in  the 
order  shown  in  Figure  7,  where  A.  is  assigned  to  Station  A,  Ah  assigned  to  B,  and  Ac  assigned 
to  C.  Therefore  station  requests  on  the  Rlink  must  have  been  in  the  order  Jt*.  (request  by 
station  A),  followed  by  Rb,  followed  by  Re-  Note  that  the  queue  in  station  B  is  inconsistent  with 
the  ordering  of  the  requests.  This  inconsistency  can  be  caused  as  demonstrated  in  Example  1 . 
After  passage  of  some  time,  the  first  slot  will  have  moved  past  station  C.  Therefore  C  will  have 
dequeued  the  entry  at  the  top  of  its  queue.  The  slots  will  be  used  by  the  stations  as  follows: 
slot  A.  used  by  station  B;  slot  Ab  used  by  station  A;  and  slot  Ac  used  by  station  C. 

Notice  from  Figure  7  that  station  B  will  use  the  slot  assigned  to  A  even  though  the  request  from 
B  is  outstanding.  Station  B’s  request  is  satisfied  by  a  slot  earlier  than  its  assigned  slot,  while 
station  A's  request  is  satisfied  by  a  slot  after  its  assigned  slot.  Station  C’s  request  is  satisfied 
by  its  assigned  slot. 

However,  if  the  unassigned  slot  shown  in  Figure  7  had  been  present  ahead  of  the  assigned 
slots,  the  following  pattern  of  slot  usage  would  have  occurred:  unassigned  slot  used  by  sta¬ 
tion  B,  slot  M  deassigned;  slot  Aa  used  by  station  A;  slot  Ab  used  by  station  C  and  slot  Ae 
deassigned. 

in  this  case,  notice  that  although  station  C  uses  Ab,  B’s  request  had  already  been  satisfied  by 
an  earlier  slot  and  hence  Ab  was  deassigned.  Requests  by  stations  B  and  C  are  satisfied  by 
slots  earlier  than  their  assigned  slots,  while  station  A’s  request  is  satisfied  by  its  assigned  slot. 

Therefore  the  behavior  of  the  system  depends  on  the  presence  and  location  of  unassigned 
slots,  in  particular  it  is  not  possible  to  predict  whether  station  A’s  request  will  be  satisfied  by 
its  assigned  slot,  an  earlier  slot,  or  a  later  slot.  This  occurs  because  the  transmission  queue  in 
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Figure  7:  Unpredictable  Behavior  of  Inconsistent  Systems 

station  B  is  inconsistent  with  the  other  station  queues.  Therefore,  if  the  queues  in  the  stations 
are  inconsistent,  the  behavior  of  the  system  is  unpredictable.  We  will  now  show  that  in  a 
coherent  multi-priority  system  ,  a  request  cannot  be  satisfied  by  a  slot  assigned  to  a  request  of 
higher  priority.  To  reason  about  equal-priority  requests  in  a  multi-priority  system,  we  introduce 
the  notion  of  effective  priority. 

Definition  7  Effective  Priority: 

Given  two  equal-priority  requests  and  Rj,  if  Ri  •<  R,  on  the  Rlink,  then  we  consider 
Ri  to  have  a  higher  effective  priority  than  that  of  Rj. 

Lemma  6  Slot  usage  patterns  caused  by  effective  priorities  are  equivalent  to  thosecaused 
by  priorities. 


Proof: 

Consider  two  requests  Ri  and  Rj.  Suppose  Ri  has  a  higher  assigned  priority  than 
that  of  Rj.  The  by  Condition  4  (priority  queues)  Ri  will  be  ahead  of  Rj  in  all  queues 
where  both  appear. 

Now  consider  Ri  and  Rj  to  have  the  same  assigned  priority  and  Ri  -<  Rj  on  the 
Rlink.  Then  by  Lemma  2,  Ri  •<  Rj  in  ail  queues  where  both  appear. 

The  slot  usage  patterns  by  requests  is  determined  by  their  relative  positions  in  sta¬ 
tion  queues.  Therefore  the  lemma  follows. 
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Theorem  7  In  a  multi-priority  coherent  system,  a  request  cannot  be  satisfied  by  a  slot 
assigned  to  another  request  with  a  higher  priority  or  higher  effective  priority. 


Proof: 

Consider  two  requests  Ri  and  Rm  such  that  R,  has  higher  priority  than  Rm-  By 
Lemma  6  this  also  covers  the  case  of  Ri  having  higher  effective  priority  than  that  of 
Rj. 

Consider  stations  S,  and  Sm  that  generated  R,  and  Rm  respectively.  It  is  sufficient 
to  consider  only  the  operation  of  Si  and  Sm  because,  as  defined  in  the  operational 
rule  of  CRP,  a  request  can  only  be  satisfied  by  the  station  that  generated  it. 

Let  the  slots  assigned  to  Ri  and  Rm  be  Ai  and  Am  respectively.  Suppose  that  R, 
is  waiting  to  be  satisfied  and  Ai  remains  to  be  an  assigned  slot,  and  that  Rm  is 
satisfied  by  A,.  We  show  that  this  is  not  possible  under  the  CRP  protocol. 

Consider  the  case  in  which  Si  and  Sm  are  in  fact  the  same  station.  Since  Rm  is 
satisfied  by  Ai  while  Ri  is  waiting  in  the  queue,  Rm  must  be  ahead  of  R,  in  the 
queue.  But  the  priority  of  Ri  is  higher  than  that  of  Rm-  This  contradicts  Condition 
4  which  states  that  station  queues  are  priority  ordered.  We  now  consider  the  case 
where  5,  and  S,  are  two  different  stations.  There  are  two  cases. 

Case  1:  Station  5m  is  upstream  with  respect  to  Si. 

Since  A,  has  been  generated,  request  R,  must  have  traveled  all  the  way  upstream 
and  reached  the  Flink  slot  generator.  Therefore  station  Sm  must  have  entered  R, 
in  its  queue.  Therefore  both  Ri  and  Rm  are  in  the  queue  of  Sm-  The  assumption 
that  Rm  is  satisfied  by  Ai  while  Ri  is  waiting  implies  lower-priority  Rm  is  ahead  of 
higher-priority  Ri  in  the  transmission  queue  of  Sm.  This  contradicts  the  assumption 
that  queues  are  priority  ordered. 

Case  2:  Si  is  upstream  with  respect  to  Sm- 

In  this  case,  the  assigned  slot  Ai  passes  station  Si  first.  In  the  following  discussion 
we  ignore  any  station  with  an  empty  transmission  queue  since  it  does  not  affect  the 
analysis. 

The  assigned  slot  Ai  can  be  used  by  station  Si  or  any  station  S  between  station  Si 
and  Sm,  unless  there  is  a  non-self  entry  request  at  the  top  of  all  their  transmission 
queues.  In  this  case,  Ai  will  be  let  go  and  each  of  these  non-self  entry  requests  are 
dequeued.  As  a  result,  Ai  will  be  available  for  station  Sm  to  use.  However,  we  show 
that  station  Sm  cannot  use  Ai  to  satisfy  R, n.  Let  Rg  be  the  non-self  entry  request 
at  the  top  of  the  queue  of  station  Si. 

We  first  establish  the  intermediate  result  that  moving  downstream  from  station  Si 
to  the  upstream  station  next  to  Sm,  Sm_,,  the  priorities  of  their  top  non-self  entry 
requests  are  non-decreasing.  We  shall  refer  to  this  result  as  the  non-decreasing 
priority  argument.  Suppose  that  this  argument  is  false  and  consider  any  pair  of 
stations  from  Si  to  Sm_  1t  say  station  and  its  next  downstream  station  5fe+1.  Let 
the  non-self  entry  requests  at  the  top  of  their  transmission  queues  be  Rk  and 
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respectively,  with  Rk  having  a  higher  priority  than  iZfc+1 .  Since  Rk  is  a  non-self  entry 
request,  it  must  have  been  generated  by  either  station  5fc+1  or  a  station  further 
downstream.  In  either  case,  request  Rk  must  appear  at  station  5fc+1’s  transmission 
queue.  Since  Rk  is  presumed  to  have  higher  priority,  it  should  be  ahead  of  Rk+ 1 . 
This  contradicts  the  assumption  that  R^  is  at  the  top  of  station  Sfc+1 's  transmission 
queue.  This  completes  the  proof  of  the  non-decreasing  priority  argument. 

We  now  prove  that  station  Sm  cannot  use  A,  to  satisfy  request  Rm-  Let  the  non-self 
entry  request  at  the  top  of  Sm-\  be  Rm-\-  Since  Rm- 1  is  a  non-self  entry  request 
at  5m_, ,  it  must  be  generated  by  either  station  Sm  or  a  station  further  down  stream. 
In  either  case,  request  Rm- 1  must  appear  at  the  transmission  queue  of  station  Sm. 

Because  of  the  non-decreasing  priority  argument,  the  priority  of  Rm-t  is  at  least  as 
high  is  the  non-self  entry  request  at  the  top  of  Si  s  transmission  queue,  request  RB. 
Since  the  priority  of  request  Rh  is  higher  than  that  of  Ri  and  the  priority  of  it,  is 
higher  than  that  of  Rm,  the  priority  of  Rm-i  is  higher  than  that  of  Rm- 

As  a  result,  Rm- 1  must  be  ahead  of  Rm  at  the  transmission  queue  of  Sm -  Hence, 
when  Ai  passes  station  Sm,  Sm  can  either 

•  Use  Ai  to  satisfy  Rm_ ,  if  l^-i  is  a  self  entry  of  station  Sm,  or 

•  Let  go  of  Ai  and  dequeue  Rm-i  if  -Rm-i  is  not  a  self  entry. 

Either  of  these  two  cases  contradicts  the  assumption  that  Rm  is  satisfied  by  A,.  □ 


Theorem  8  For  any  periodic  connection  in  a  dual-link  network  that  follows  CRP,  the  max¬ 
imum  duration  of  priority  inversion  is  bounded  by  2di  where  di  is  the  distance  in  slot 
times  between  the  source  station  and  the  Flink  slot  generator. 


Proof: 

Suppose  there  is  a  low-priority  connection  tl  at  the  head  station  that  occupies  every 
Flink  slot. 

Consider  a  connection  tb  of  higher  priority  than  tl.  Let  tb  generate  a  request  at 
time  t»t0.  The  request  of  connection  th  cannot  be  delayed  on  the  Rlink  by  lower- 
priority  requests  due  to  the  request  preemption  rule  of  CRP.  However,  since  all  Flink 
slots  are  being  used  by  tl,  tb  is  prevented  from  transmission.  Excluding  effects 
of  preemption  on  the  Rlink,  the  request  of  tb  will  reach  the  Flink  slot  generator  at 
time  *<>  +  di.  An  Flink  slot  will  be  assigned  to  tb.  By  Theorem  7,  tl  cannot  use  this 
assigned  slot.  With  an  additional  delay  of  di,  the  assigned  slot  will  arrive  at  th’s 
station  and  can  be  used  by  tb. 

Therefore  after  tb  generates  a  request  it  can  be  delayed  by  lower-priority  connec¬ 
tions  for  a  maximum  of  2 di  slots.  The  theorem  follows.  □ 
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Theorem  9  A  dual-link  network  that  follows  CRP  is  coherent. 


Proof: 


This  theorem  follows  because  of  Theorem  5  and  Theorem  8. 


□ 


5  Scheduling  Dual-Link  Networks 


In  this  section  we  investigate  the  use  of  a  coherent  dual-link  network  for  scheduling  periodic 
real-time  traffic.  We  focus  on  periodic  traffic  scheduling  for  the  following  reasons: 

•  Voice  and  video  traffic  sources  are  periodic  in  nature.  Even  compressed  video  may  be 
periodic,  since  practical  VLSI  compression  devices,  at  least  those  for  MPEG,  [Gal91]  and 
Px64,  [Lio91],  may  have  "rate-control”  buffers,  so  that  the  compressed-video  output  is  at 
a  constant  data  rate. 

•  Traditional  real-time  applications  generate  periodic  traffic  from  sampled  data  systems. 
Although  aperiodic  real-time  traffic  may  exist  in  the  network,  it  can  be  handled  by  aperiodic 
server  algorithms,  e.g.,  the  sporadic  server  [Spr90]  or  the  deferrable  server  algorithm  as 
demonstrated  by  Strosnider  [str88],  which  can  be  analyzed  as  if  it  is  periodic. 

•  Non-real-time  traditional  aperiodic  traffic  such  as  interactive  data  processing.  File  trans¬ 
fers  can  be  given  either  an  aperiodic  server  or  served  at  background  priority. 

Scheduling  dual-link  networks  is  different  from  scheduling  a  centralized  system,  since  some 
requests  are  never  seen  oy  some  stations.  Hence  we  cannot  directly  use  scheduling  results 
from  centralized  systems.  Nonetheless,  we  will  show  that  if  a  set  of  connections  is  schedulable 
in  a  centralized  system,  it  is  also  schedulable  in  a  dual-link  network,  allowing  for  initial  delay. 
We  will  call  periodic  traffic  between  a  source  station  and  destination  station  a  connection  in  the 
rest  of  the  paper.  Each  connection  wishes  to  transmit  a  message  of  Ci  fixed-size  packets  per 
period  Tt.  Packet  size  is  same  as  the  slot  size  on  the  network  links.  We  assume  that  the  time 
to  transmit  each  slot  is  unity,  and  that  each  connection’s  period  is  assumed  to  be  an  integral 
number  of  slot  transmission  times. 

Consider  a  set  of  periodic  connections  r1,r2,... rn  arranged  in  decreasing  priority  order.  We 
are  interested  in  the  worst-case  delay  for  a  periodic  connection.  We  first  show  the  equivalence 
between  relative  results  when  its  request  is  delayed  by  all  higher-priority  requests.  A  useful 
lemma  in  centralized  system  scheduling  is  the  critical  instant  Lemma  10  [LL73]. 


Lemma  10  Given  a  set  of  periodic  activities  in  a  centralized  system,  the  longest  com¬ 
pletion  time  for  any  activity  occurs  when  it  is  initiated  at  the  critical  instant.  The  critical 
instant  is  the  time  at  which  a  task  is  initiated  along  with  all  tasks  of  higher  priority. 
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Lemma  11  Consider  two  connections  th  and  tl  arranged  in  decreasing  priority  order. 
LetrB  and  tl  be  in  stations  Sh  and  SL  respectively.  Let  propagation  delay  between  Sl 
and  Sb  bo  dm-  The  preemption  effect  on  the  tl  can  be  modeled  as  though  th  is  in  the 
same  station  as  tl  with  starting  times  modified  as  follows:  if  Sb  is  downstream  to  Sl 
then  dsL  is  added  to  the  starting  time  ofrB .  Otherwise  it  is  subtracted  from  the  starting 
time  of  tb. 

Proof: 

Case  1:  tb  is  in  a  downstream  station  SB. 

t l  will  experience  preemption  from  th  after  tQ  +  k  +  dLH.  This  is  equivalent  to 
having  th  in  station  Sl  but  starting  after  time  t0  +  Jb  +  dLs- 

Case  2:  tb  is  in  an  upstream  station  Sh- 

tl  will  experience  preemption  due  to  tb  after  t0  +  k  -  dLn-  This  is  equivalent  to 
having  tb  in  station  Sl  but  starting  after  time  f0  +  k-  dLB- 

Lemma  12  Given  a  set  of  period  connections  in  a  dual-link  network,  the  longest  delay 
experienced  by  any  request  initiated  at  time  t=0  is  no  greater  than  the  delay  that  results 
when  all  equal-  or  higher-priority  connections  are  located  in  the  same  station  and  gen¬ 
erate  requests  at  time  t- 0. 

Proof: 

For  any  given  connection  t{  at  station  Sx,  move  all  higher-priority  connections  into 
Si  using  the  transformation  technique  of  Lemma  1 1 .  This  preserves  the  preemp¬ 
tion  effects  on  tj.  Since  r,  and  all  higher-priority  connections  are  now  in  the  same 
station,  the  scheduling  problem  is  a  centralized  one.  Under  this  condition,  Lemma 
10  applies.  The  lemma  follows. 

Lemma  1 3  Consider  a  set  of  n  connections  ^ ,  r2>  •  •  •  rn  arranged  in  decreasing  priority 
order.  In  a  dual-link  network  under  CRP  and  the  flow  control  protocol,  a  request  can 
only  be  satisfied  by  its  assigned  slot. 

Proof: 

Consider  any  connection  r,  from  station  5,,  that  makes  <7,  requests  every  period  Tt. 

Let  each  request  be  denoted  Ri  and  the  corresponding  assigned  slot  be  denoted 
Ai.  There  are  only  two  cases. 

Case  1:  Request  Ri  from  r{  is  not  preempted  before  reaching  the  Flink  slot  gener¬ 
ator. 

In  this  case  Ri  is  not  delayed  by  preemptions  and  the  slot  A,  will  arrive  at  Si  exactly 
2<L  time  units  later.  By  the  flow  control  protocol,  At  will  be  used  by  r,  unless  it  has 
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been  used  eariier  by  some  other  station.  We  show  that  Ai  cannot  be  used  earlier 
by  any  other  station. 

There  are  three  subcases  to  be  considered: 


Case  la:  t{  is  the  highest-priority  connection. 

In  this  case  by  Theorem  7,  A,  cannot  be  used  by  any  other  connection. 

By  the  flow  control  protocol,  r,  will  not  be  ready  to  transmit  until  the  first  A; 
arrives  at  S{.  Therefore  r<  will  use  the  first  Ci  assigned  slots  (A,).  Further, 
since  r,  is  not  preempted,  Ci  assigned  slots  will  arrive  at  5,  exactly  one 
period  T,  apart  and  will  be  used  by  n. 

Case  1b:  t{  is  not  the  highest-priority  connection  and  higher-priority  con¬ 
nection  rh  from  station  Sh  makes  a  request  Rh  such  that  Rh  <  Ri. 

Ah,  (slot  assigned  to  Rh)  will  arrive  at  the  source  of  connection  t*  before 
A,  arrives  at  the  source  of  t*.  Further,  Ah  arrives  at  Sh  exactly  2 dh  units 
later.  Hence  by  the  flow  control  protocol,  rh  uses  Ah.  Therefore  rh  cannot 
use  Ai. 

Case  1c:  r;  is  not  the  highest-priority  connection  and  higher-priority  con¬ 
nection  Th  makes  a  request  Rh  such  that  R*  -<  Rh. 

In  this  case  Ai  -<  Ah,  and  therefore  Ai  will  arrive  at  the  source  of  connec¬ 
tion  Th  before  the  arrival  of  Ah.  However  rh  will  not  use  Ai  since  it  will  not 
be  ready  to  transmit  at  this  time  due  to  the  flow  control  protocol. 

Case  2:  Request  from  n  is  preempted  by  high-priority  requests. 

Consider  a  connection  Th  which  is  higher  priority  than  t{  and  has  Ch  packets  to 
transmit  every  Th.  Let  the  requests  by  tk  be  denoted  as  Rh. 

Let  Rh  preempt  R*.  Therefore  Rh  x  Ri  on  the  Rlink  and  Ah  <  Ai  on  the  Flink. 

By  Theorem  7,  connection  cannot  use  a  slot  assigned  to  a  high-priority  request. 
Therefore  cannot  use  Ah.  By  the  flow  control  protocol,  Th  is  ready  to  use  Ah  when 
it  arrives  at  the  source  of  Th,  and  cannot  use  more  than  Ch  per  period.  Therefore  t* 
will  use  its  assigned  slots.  Also  by  the  flow  control  protocol,  r*  is  ready  by  the  time 
Ai  arrives. 

Therefore  each  Ri  can  only  be  satisfied  by  A<. 

□ 

Because  of  the  potentially  long  propagation  delay  in  wide  area  networks,  the  traditional  notion 
of  schedulability  needs  to  be  extended  to  take  the  propagation  delay  into  account.  We  introduce 
the  notion  of  transmission  schedulability. 


Definition  8  Transmission  Schedulability: 

A  connection  t{  is  said  to  be  transmission  schedulable,  ( t-schedulable )  if  it  can  transmit 
Ci  packets  per  period  T„  after  an  initial  delay  bounded  by  2d,  +  T„  where  d,  is  the  prop¬ 
agation  between  the  connection’s  station  and  the  head  station. 
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Theorem  14  Given  a  set  of  periodic  connections,  if  the  set  of  periodic  connections  is 
schedulable  in  a  centralized  preemptive  priority-driven  system  with  zero  propagation 
delay,  then  the  set  of  connections  is  t-schedulable  in  a  dual-link  network. 

Proof: 

Since  we  have  shown  that  the  worst-case  preemption  delay  experienced  by  a  re¬ 
quest  in  a  dual-link  network  is  same  as  the  delay  experienced  in  a  centralized  sys¬ 
tem  with  the  same  connection  set,  if  the  connection  set  is  schedulable  in  a  cen¬ 
tralized  system,  then  each  connection  will  be  able  to  make  C,  requests  every  T<. 
Therefore  the  Flink  slot  generator  will  receive  C,  requests  every  T<  from  connection 
Ti  after  an  initial  delay  bound  by  <k  +  T,.  The  Flink  slot  generator  will  therefore  as¬ 
sign  C,  slots  to  ^  every  period  T,  after  this  initial  delay.  By  Lemma  1 3,  connection  r, 
can  always  use  its  assigned  slots.  Therefore,  it  will  be  able  to  transmit  its  message 
every  period  after  a  delay  bound  by  2d,  +  Ti.  Therefore  the  theorem  follows.  □ 

Theorem  15  In  a  t-schedulable  coherent  dual-link  network,  a  connection  r  with  C  pack¬ 
ets  to  transmit  per  period  T  will  require  C\2dijT ]  buffers  in  the  source  station  of  the 
connection. 

Proof: 

By  the  flow  control  protocol,  the  source  of  the  connection  will  not  transmit  until 
2 di  slot  times  after  the  arrival  of  the  first  request.  In  this  time  C\2di/T~\  packets 
will  arrive  at  the  station  for  connection  r  and  must  be  buffered.  Therefore  at  least 
C\2di/T ]  buffers  will  be  necessary. 

Since  the  network  is  t-schedulable,  the  source  station  will  be  able  to  transmit  C 
packets  every  T  after  the  initial  delay  of  24,-.  Therefore  the  packet  arrival  rate  at 
the  source  station  is  equal  to  the  transmission  rate  and  additional  buffers  are  not 
necessary. 

Hence  the  theorem  follows.  □ 

Given  a  set  of  connections  that  is  t-schedulable,  the  end-to-end  delay  experienced  by  a  mes¬ 
sage  of  any  connection  t,  is  given  by 

End-to-end  Delay=24,  +  T,  +  Dpropi i,t) 

where  iVop(t,0  is  the  propagation  delay  between  the  source  station  Si  and  the  destination 
station  St  of  connection  r,. 


6  Engineering  Considerations 

In  this  paper  we  have  developed  a  model  of  a  dual-link  network  which  allows  us  to  achieve  a 
high  degree  of  schedulability  and  exhibits  predictable  timing  behavior.  In  this  section  we  first 
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discuss  implementation  considerations  for  a  dual-link  network  architecture.  We  then  briefly 
compare  our  implementation  with  IEEE  802.6. 


6.1  Implementation  Considerations 

The  dual-link  network  abstraction  in  previous  sections  was  designed  to  facilitate  analysis.  We 
now  reconsider  this  model  from  an  implementation  standpoint.  First,  station  queues  can  be 
replaced  by  a  set  of  counters  similar  to  those  in  IEEE  802.6  [Sta90].  We  considered  Flink  slots 
to  consist  of  the  BUSY  bit  and  data,  and  Rlink  slots  to  consist  of  the  REQ  bit,  and  the  priority 
field.  Since  each  link  is  actually  used  for  reservation  of  the  opposite  link  and  transmission  of 
data,  slots  on  each  link  should  consist  of  the  BUSY  bit,  data,  a  REQ  bit,  and  the  priority  field. 
A  further  optimization  might  be  to  omit  the  REQ  bit  and  let  the  zero  value  in  the  priority  field 
denote  the  lack  of  a  request. 

We  now  discuss  the  priority  field.  A  significant  aspect  of  priority-based  scheduling  in  real-time 
systems  is  the  number  of  priorities  that  should  be  supported  by  the  arbitration  logic.  Ideally, 
there  should  be  as  many  priority  levels  as  the  different  connection  periods.  When  the  priority 
levels  are  fewer  than  the  number  of  different  periods,  schedulability  is  reduced  as  discussed  in 
[SRL91].  In  this  paper  our  t-schedu lability  definition  requires  that  each  connection  must  be  able 
to  transmit  one  message  every  period.  This  is  equivalent  to  centralized  scheduling  in  which 
each  periodic  activity  must  meet  its  end-of-period  deadline.  [SRL91]  shows  that  the  schedula¬ 
bility  loss  is  negligible  with  256  priority  levels.  Ideally  a  dual-link  network  for  real-time  applica¬ 
tions  should  have  256  priority  levels,  although  it  may  be  possible  to  meet  the  t-schedulability 
requirement  with  fewer  priority  levels,  depending  on  the  characteristics  of  connections  in  the 
network. 

We  proposed  a  protocol  in  which  priority  is  implemented  as  an  8-bit  encoded  field  to  yield  256 
priority  levels.  This  protocol  allows  preemption  of  lower-priority  slots  by  higher-priority  slots. 
Each  slot  contains  an  8-bit  encoded  priority  field  in  the  header.  Higher  numbers  are  used  to 
indicate  higher  priorities.  All  zeros  in  the  priority  field  can  indicate  the  absence  of  a  request.  A 
station  that  wants  to  make  a  request  at  priority  i  behaves  as  follows: 


If  the  next  slot  received  contains  a  request  at  priority  j,  then 

•  If  j  >  t,  the  station  waits  for  the  next  slot. 

•  If  j  <  *,  the  station  replaces  the  priority  field  j  of  the  slot  with  *  and  stores  j  in 
a  prioritized  request  queue. 

Therefore  the  station  preempts  lower-priority  reservations  with  higher-priority  reser¬ 
vations. 


The  advantage  of  the  encoded  priority  field  is  that  it  allows  the  implementation  of  256  priority 
levels  with  only  8  bits  of  overhead  in  each  slot.  A  simplified  implementation  of  request  pre¬ 
emption  logic  is  shown  in  Figure  8.  The  slot  priority  from  the  link  is  passed  through  a  single-bit 
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Shift  Roglatar 
hoM«  atation  priority 


Figure  8:  Proposed  Request  Preemption  Circuit 


delay  and  compared  bit  by  bit  with  the  station  priority  that  is  stored  in  the  shift  register.  As  long 
as  the  priority  bits  match,  the  output  of  the  exdusive-OR  gate  is  zero  and  the  link  priority  is 
output.  As  soon  as  the  priority  bits  differ,  the  station  priority  bits  are  output  if  they  have  a  higher 
priority.  Otherwise  the  link  priority  bits  continue  to  be  transmitted.  Note  that  the  logic  assumes 
that  the  most  significant  priority  bite  in  the  slot  are  received  first. 


6.2  Implications  to  IEEE  802.6 


There  are  two  main  functional  differences  between  our  dual-link  network  model  and  IEEE  802.6 
DQDB.  First,  in  an  attempt  to  achieve  fairness,  in  IEEE  802.6  a  station  cannot  make  a  new 
request  on  the  Rlink  if  its  previous  request  is  outstanding.  This  makes  the  request  traffic  non- 
autonomous  and  dependent  on  the  traffic  on  the  Flink.  As  we  have  shown,  this  may  result 
in  unbounded  priority  inversion  and  make  the  system  incoherent.  The  second  less  serious 
difference  is  that  a  station  in  IEEE  802.6  can  use  a  slot  on  the  Flink  before  making  a  request  on 
the  Rlink,  provided  its  CD  counter  is  zero.  This  is  acceptable  when  the  system  is  schedulable. 
When  the  system  is  overloaded,  it  is  not  possible  to  predict  which  station  will  miss  deadlines. 

IEEE  802.6  implements  priority  by  having  a  separate  REQ  bit  for  each  priority  level.  Because 
of  this  implementation,  it  is  not  possible  to  implement  a  large  number  of  priority  levels  with¬ 
out  excessive  overhead.  Hence  IEEE  802.6  implements  only  4  priority  levels.  As  we  have 
shown,  this  may  not  be  sufficient  and  may  result  in  low  schedulable  utilization,  depending  on 
the  characteristics  of  the  connections  in  the  network. 
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7  Conclusions  and  Future  Work 


We  have  developed  a  general  model  of  reservation-based  dual-link  networks  and  used  it  to 
reason  about  the  relationship  between  station  request  patterns  and  slot  usage  patterns.  We 
introduced  the  concept  of  system  coherence  and  examined  the  properties  of  coherent  sys¬ 
tems.  We  showed  that  a  coherent  dual-link  network  can  be  analyzed  similarly  to  an  equivalent 
centralized  system  in  terms  of  its  schedulability  for  periodic  message  traffic. 

A  number  of  important  issues  remain  to  be  addressed. 

Bandwidth  allocation  and  overload  management:  Overload  management  is  a  challenging 
problem  in  a  metropolitan  area  network  because  scheduling  decisions  are  made  in  a 
distributed  manner.  Nevertheless,  we  must  have  the  the  ability  to  specify  an  arbitrary 
subset  of  traffic  sources  that  meet  deadlines  even  under  overload. 

Integration  between  periodic  and  aperiodic  messages:  We  need  to  extend  this  analysis  to 
address  both  periodic  and  aperiodic  traffic  in  a  unified  framework. 

Effect  of  introducing  erasure  nodes:  When  a  station  receives  a  packet,  the  slot,  in  principle, 
can  be  "erased”  and  be  used  again.  The  use  of  erasure  nodes  and  their  effect  on  network 
predictability  needs  to  be  considered. 
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